How Perceptual Categories Influence Trial and Error Learning in Humans

Converging evidence suggests that trial-and-error learning in humans shares many computational principals with contemporary RL algorithms (Montague, Hyman, & Cohen, 2004; Frank, Seeberger, & O’Reilly, 2004). However, a critical feature of these algorithms is the notion of “state.” State representations help structure which actions to take in particular situations, as well as how to assign credit for delayed rewards. In human reinforcement learning research, the issue of state or context is often avoided by simply assuming that states are interchangeable with the notion of stimulus. Changes in state are simply changes in the currently perceived stimulus. However, the association of states with distinct stimuli is problematic since it limits generalization between different experiences and leads to a combinatorial explosion of potential aspects of the world one must learn about. As a result, human (and artificial) learners must often adopt more nuanced representations that integrate across multiple cues and allow generalization between similar situations. For example, in predicting whether a new restaurant will offer a good meal, many features regarding its decoration, menu, and location may be relevant. These cues combine to allow one to estimate the desirability of the new experience given previous experiences. In addition, a good dining experience at one restaurant may cause changes in the valuation of other, similar restaurants.